The influence of selection bias on effort overruns in software development projects

Context: A potentially important, but neglected, reason for effort overruns in software projects is related to selection bias. Selection bias–induced effort overruns occur when proposals are more likely to be accepted and lead to actual projects when based on effort estimates that are too low rather than on realistic estimates or estimates that are too high. The effect of this bias may be particularly important in bidding rounds, but is potentially relevant in all situations where there is effort or costbased selection between alternatives. Objective: To better understand the relevance and management of selection bias effects in software development contexts. Method: First, we present a statistical model illustrating the relation between selection bias in bidding and other contexts and effort overruns. Then, we examine this relation in an experiment with software professionals who estimated and completed a set of development tasks and examine relevant field study evidence. Finally, we use a selection bias scenario to assess awareness of the effect of selection bias among software providers. Results: The results from the statistical model and the experiment demonstrated that selection bias is capable of explaining much of the effort overruns. The field evidence was also consistent with a substantial effect of selection bias on effort overruns, although there are alternative explanations for the findings. We found a low awareness of selection bias among the software providers. Conclusion: Selection bias is likely to be an important source of effort overruns and should be addressed to reduce problems related to over-optimistic effort estimates.